Four questions to guide decision-making for data sharing and integration.

Introduction: This paper presents a Four Question Framework to guide data integration partners in building a strong governance and legal foundation to support ethical data use. Objectives: While this framework was developed based on work in the United States that routinely integrates public data, it is meant to be a simple, digestible tool that can be adapted to any context. Methods: The framework was developed through a series of public deliberation workgroups and 15 years of field experience working with a diversity of data integration efforts across the United States. Results: The Four Questions-Is this legal? Is this ethical? Is this a good idea? How do we know (and who decides)?-should be considered within an established data governance framework and alongside core partners to determine whether and how to move forward when building an Integrated Data System (IDS) and also at each stage of a specific data project. We discuss these questions in depth, with a particular focus on the role of governance in establishing legal and ethical data use. In addition, we provide example data governance structures from two IDS sites and hypothetical scenarios that illustrate key considerations for the Four Question Framework. Conclusions: A robust governance process is essential for determining whether data sharing and integration is legal, ethical, and a good idea within the local context. This process is iterative and as relational as it is technical, which means authentic collaboration across partners should be prioritized at each stage of a data use project. The Four Questions serve as a guide for determining whether to undertake data sharing and integration and should be regularly revisited throughout the life of a project. Highlights: Strong data governance has five qualities: it is purpose-, value-, and principle-driven; strategically located; collaborative; iterative; and transparent.Through a series of public deliberation workgroups and 15 years of field experience, we developed a Four Question Framework to determine whether and how to move forward with building an IDS and at each stage of a data sharing and integration project.The Four Questions-Is this legal? Is this ethical? Is this a good idea? How do we know (and who decides)?-should be carefully considered within established data governance processes and among core partners.

Leveraging integrated data for program evaluation: Recommendations from the field.

Use of administrative data to inform decision making is now commonplace throughout the public sector, including program and policy evaluation. While reuse of these data can reduce costs, improve methodologies, and shorten timelines, challenges remain. This article informs evaluators about the growing field of Integrated Data Systems (IDS), and how to leverage cross-sector administrative data in evaluation work. This article is informed by three sources: a survey of current data integration efforts in the United States (U.S.) (N=63), informational interviews with experts, and internal knowledge cultivated through Actionable Intelligence for Social Policy's (AISP) 12+ years of work in the field. A brief discussion of the U.S. data integration context and history is provided, followed by discussion of tangible recommendations for evaluators, examples of evaluations relying on integrated data, and a list of U.S. IDS sites with publicly available processes for external data requests. Despite the challenges associated with reusing administrative data for program evaluation, IDS offer evaluators a new set of tools for leveraging data across institutional silos.

Evidence-Based Policymaking: What Human Service Agencies Can Learn from Implementation Science and Integrated Data Systems.

The evidence-based policymaking movement compels government leaders and agencies to rely on the best available research evidence to inform policy and program decisions, yet how to do this effectively remains a challenge. This paper demonstrates how the core concepts from two emerging fields-Implementation Science (IS) and Integrated Data Systems (IDS)-can help human service agencies and their partners realize the aims of the evidence-based policymaking movement. An IS lens can help agencies address the role of context when implementing evidence-based practices, complement other quality and process improvement efforts, simultaneously study implementation and effectiveness outcomes, and guide de-implementation of ineffective policies. The IDS approach offers governance frameworks to support ethical and legal data use, provides high-quality administrative data for in-house analyses, and allows for more time-sensitive analyses of pressing agency needs. Ultimately, IS and IDS can support human service agencies in more efficiently using government resources to deliver the best available programs and policies to the communities they serve. Although this paper focuses on examples within the United States context, key concepts and guidance are intended to be broadly applicable across geographies, given that IS, IDS, and the evidence-based policymaking movement are globally relevant.

Developmental impacts of the COVID-19 pandemic on young children: a conceptual model for research with integrated administrative data systems.

The COVID-19 pandemic made its mark on the entire world, upending economies, shifting work and education, and exposing deeply rooted inequities. A particularly vulnerable, yet less studied population includes our youngest children, ages zero to five, whose proximal and distal contexts have been exponentially affected with unknown impacts on health, education, and social-emotional well-being. Integrated administrative data systems could be important tools for understanding these impacts. This article has three aims to guide research on the impacts of COVID-19 for this critical population using integrated data systems (IDS). First, it presents a conceptual data model informed by developmental-ecological theory and epidemiological frameworks to study young children. This data model presents five developmental resilience pathways (i.e. early learning, safe and nurturing families, health, housing, and financial/employment) that include direct and indirect influencers related to COVID-19 impacts and the contexts and community supports that can affect outcomes. Second, the article outlines administrative datasets with relevant indicators that are commonly collected, could be integrated at the individual level, and include relevant linkages between children and families to facilitate research using the conceptual data model. Third, this paper provides specific considerations for research using the conceptual data model that acknowledge the highly-localised political response to COVID-19 in the US. It concludes with a call to action for the population data science community to use and expand IDS capacities to better understand the intermediate and long-term impacts of this pandemic on young children.

A framework for centering racial equity throughout the administrative data life cycle.

INTRODUCTION: Data integration by local and state governments is undertaken for the public good to support the interconnected needs of families and communities. Though data infrastructure is a powerful tool to support equity-oriented reforms, racial equity is rarely centered or prioritized as a core goal for data integration. This raises fundamental concerns, as integrated data increasingly provide the raw materials for evaluation, research, and risk modeling. Generally, institutions have not adequately examined and acknowledged structural bias in their history, or the ways in which data reflect systemic racial inequities in the development and administration of policies and programs. Meanwhile, civic data users and the public are rarely consulted in the development and use of data systems. OBJECTIVES: This paper presents a framework and site-based examples of "Work in Action" that were collaboratively generated by a civic data stakeholder workgroup from across the U.S. in 2019-2020. METHODS: Purposive sampling was used to curate a diverse 15-person workgroup that used participatory action research and public deliberation to co-create a framework of best practices. RESULTS: This framework aims to support agencies seeking to acknowledge and compensate for the harms and bias baked into data and practice. It is organized across six stages of the administrative data life cycle-planning, data collection, data access, use of algorithms/statistical tools, analysis, and reporting and dissemination. For each stage, the framework includes positive and problematic practices for centering racial equity, with site-based examples of "Work in Action" from across the U.S. Using this framework, the workgroup then developed a Toolkit for Centering Racial Equity Throughout Data Integration, a resource that has been broadly disseminated across the U.S. CONCLUSIONS: Findings indicate that centering racial equity within data integration efforts is not a binary outcome, but rather a series of small steps towards more equitable practice. There are countless ways to center racial equity across the data life cycle, and this framework provides concrete strategies for organizations to begin to grow that work in practice.